Active inference unifies intentional and conflict-resolution imperatives of motor control

The field of motor control has long focused on the achievement of external goals through action (e.g., reaching and grasping objects). However, recent studies in conditions of multisensory conflict, such as when a subject experiences the rubber hand illusion or embodies an avatar in virtual reality, reveal the presence of unconscious movements that are not goal-directed, but rather aim at resolving multisensory conflicts; for example, by aligning the position of a person’s arm with that of an embodied avatar. This second, conflict-resolution imperative of movement control did not emerge in classical studies of motor adaptation and online corrections, which did not allow movements to reduce the conflicts; and has been largely ignored so far in formal theories. Here, we propose a model of movement control grounded in the theory of active inference that integrates intentional and conflict-resolution imperatives. We present three simulations showing that the active inference model is able to characterize movements guided by the intention to achieve an external goal, by the necessity to resolve multisensory conflict, or both. Furthermore, our simulations reveal a fundamental difference between the (active) inference underlying intentional and conflict-resolution imperatives by showing that it is driven by two different (model and sensory) kinds of prediction errors. Finally, our simulations show that when movement is only guided by conflict resolution, the model incorrectly infers that is velocity is zero, as if it was not moving. This result suggests a novel speculative explanation for the fact that people are unaware of their subtle compensatory movements to avoid multisensory conflict. Furthermore, it can potentially help shed light on deficits of motor awareness that arise in psychopathological conditions.

In this section, we briefly summarize the mathematical formulation of the free energy principle and the ensuing active inference formulation of perception and action. Please see [1][2][3] for additional details.
The free energy principle has been formulated to provide a unifying theory of perception and action in biological organisms and their associated neuronal underpinnings. At its root the free energy principle embraces the Helmholtzian perspective, according to which perception, action and other cognitive processes are the result of a probabilistic inference about the hidden causes of sensory signals. In the classical Bayesian Inference formulation, this process is equivalent to computing the posterior probability of the hidden causes, , given the sensory signals available, , as give by the Bayes rule:

Here
represents the likelihood of the sensory signals for a given configuration of the environment (and thus of the hidden causes ) an represents the prior expectation on the state of the environment that the agent holds before receiving sensory information. The normalization factor at the denominator, , represents the marginal likelihood (obtained marginalizing the likelihood over all possible states), which is in most applications is intractable.
Variational free energy [4] has been introduced as an approximation of Bayesian Inference problems that require computing the posterior probability of the hidden causes, . The variational approximation is based on the idea that an inferring agent holds an approximate representation of the posterior probability , known as the recognition density, , which can be computed by minimizing the Kullback-Leibler (KL) divergence between the two distributions: Note that the latter equivalence corresponds to the formal definition of KL divergence, which quantifies the difference between two probability distributions. Although the KL divergence as defined above cannot be derived exactly, it can be rewritten as: Where the last equivalence results from the formal definition of the variation free energy ℱ (first terms in the central and left sides), and to the fact that the logarithm in the second integral can be rewritten as log and kept out of the integration (the remaining integral of the recognition density over all possible states is 1 by definition of probability distribution). From the last equation, it appears clear that minimizing free energy with respect to the recognition density is equivalent to minimize the KL divergence between the posterior probability and (as the second term in the r.h.s of the last equation is a constant), and therefore to optimize the inference problem. The advantage of introducing variational free energy is that under a set of plausible assumptions, it is possible to derive an analytical formulation of ℱ, which could be then minimized through gradient descent methods.
By definition, the variational free energy can be rewritten as: In principle, could take any form, but in order to keep the mathematical problem tractable, q(x) is assumed to be a multivariate Gaussian specified by its sufficient statistics (i.e., by the expected means and standard deviations [ ! , Σ ! ], with ∈ 1, spanning the variables that describe the system state). With the additional assumptions that the joint probability is a smooth function of and that its logarithm can be approximated with a quadratic function near the mode, the variational free energy can be expressed in terms of the Laplace encoded energy ( , ) ≔ − log , : The two assumptions above are also known as the Laplace approximation, and the full derivation of the resulting simplified expression of ℱ have been demonstrated in previous works [5][6][7].
If one further assumes that Σ ! are constant, minimizing ℱ with respect to the recognition density is equivalent to minimizing ℱ, and therefore the Laplace encoded energy , , with respect to ! . The second term was thus, in the mathematical model description in Fig 5 referred to the constant C. The variational free energy can be then expressed by working out a treatable (derivable) expression of the Laplace-encoded energy − and of the joint probability , − where the inferred state of the environment is approximated with the sufficient statistics of the recognition density. For the problem treated in this paper and the related model implementation, this is done throughout the equations in Fig  5. Once the approximated formulations for the variational free energy have been derived, the problem of perceptual inference can be approached computationally by estimating the value of that minimizes ℱ , and therefore maximizes the joint probability of sensory data and associated internal state estimate (by definition of Laplace encoded energy), which is akin to Bayesian formulations of perception. In addition, the active inference formulation postulates that agents can minimize the free energy (and therefore maximize , ) by performing actions that affect the sensory state and can thus drive the free energy along a gradient descent. The detailed implementation of the action-perception loop adopted for our model is described in Figure 5. Crucially, the perceptual inference driven by the integrated effect of the internal system dynamics and the free energy minimization has a direct impact on the actions that are generated internally as a strategy to further minimize free energy. The actions in turn, eventually affect the dynamics of the environment.
As discussed in the main text, computing actions as part of the internal model dynamics requires an additional caveat. In fact, the internal model of the system, and therefore the expression derived for the variational free energy, do not have an explicit representation of the action. For this, active inference formulations assume that agent holds an implicit knowledge about how its actions affect the sensory state. This means that in order to select actions that minimize the free energy, the impact of action on , should be computed indirectly, by considering how actions affect the sensory states and how the sensory states affect the joint probability , . An example of this, for the specific case of the model implemented in this study, is provided in equations G.3 and G.4 of Fig 6.